Method of picture-in-picture for multimedia applications

ABSTRACT

A method and a device for preparing a media stream containing more than one component stream for picture-in-picture applications are described along with the corresponding method and device for rendering the media stream in a picture-in-picture mode. The invention allows for live and dynamic picture-in-picture rendering of the component streams contained in the media stream. Extensions to the media formats that shall support various multimedia applications, such as the MPEG-2 Transport Stream and ISO media format, are proposed to enable such a dynamic picture-in-picture functionality.

CROSS-REFERENCE TO RELATED APPLICATIONS

This application claims the benefit, under 35 U.S.C. §365 ofInternational Application PCT/US2011/043494, filed Jul. 11, 2011, whichwas published in accordance with PCT Article 21(2) on Jan. 19, 2012 inEnglish and which claims the benefit of U.S. provisional patentapplication No. 61/363,699, filed Jul. 13, 2010 and U.S. provisionalpatent application No. 61/363,697, filed Jul. 13, 2010.

TECHNICAL FIELD

The present invention relates generally to picture-in-picture formultimedia applications such as transmission and rendering. Morespecifically, the invention relates to support of picture-in-picturefunctionality and improvement of the flexibility of picture-in-picturefor multimedia applications.

BACKGROUND OF THE INVENTION

Picture-in-picture (PIP) allows viewers to view multiple separate videosources simultaneously. For example, some Blu-ray Disc titles include apicture-in-picture track that allows the viewer to see the director'scomment on a film they are watching. Traditionally, such as in Blu-rayDisc applications, the picture-in-picture is implemented by generating ahard coded PIP video, i.e. by replacing the regions in the backgroundvideo with a foreground video. The hard coded PIP video is compressedand transmitted to the receiver. As a result, viewers are not able todynamically adjust the PIP, such as to enable/disable the PIP feature(unless a copy of the background video is sent separately), to changethe position of the foreground video, etc. Another traditional PIPapplication is to overlay two independent video streams at the playerside, where video transport cannot provide any correlation informationof the PIP video streams.

With the development of interactive media technology, multiple videocomponents can be correlated and form a set of media, i.e. PIP media.The rendering of the PIP media can be dynamic, which means the position,scaling and alpha blending of the foreground videos can vary duringplayback, determined by either content creation or user interactions. Inthe previous example wherein a foreground video shows a directorcommenting on the background video, dynamic PIP enables the effect thatthe director points to different positions of the background video bye.g. moving the foreground video.

One deficiency of the current media file formats, such as MPEG2transport stream and ISO media file format, is that they cannot live ordynamically update the information of position, layer and scaling forthe PIP stream in the system layer (i.e. the transport layer). Withoutthe dynamic position and scaling information, it is not possible toreliably fit or overlay a video source on a display region that does notshare the same resolution. One possibility is to retrieve theinformation from the video decoder. Depending on the codec and theoutput of a particular encoder, such information may not exist or maynot be reliable. It may also be difficult for a system to extract thisinformation from the codec as well. A system-level approach is a betterapproach to offering a consistent experience regardless of theunderlying video codec used.

The present invention provides solutions to support picture-in-picturefunctionality and improve the flexibility of picture-in-picture formultimedia applications such as transmission and rendering.

SUMMARY OF THE INVENTION

This invention directs to methods and apparatuses for preparing a mediastream containing a plurality of component streams forpicture-in-picture application and for rendering such a media stream ina picture-in-picture mode.

According to an aspect of the present invention, there is provided amethod for preparing a media stream containing a plurality of componentstreams for picture-in-picture applications. Picture-in-pictureinformation is generated based on the plurality of component streamscontained in the media stream. Such picture-in-picture information isembedded into the media stream before sending to the receiver. Anapparatus for performing such a method is also provided. The apparatuscomprises a correlator, which generates picture-in-picture informationfrom the plurality of component streams contained in the media stream,and an embedder for inserting the picture-in-picture information intothe media stream.

According to another aspect of the present invention, there is provideda method for rendering a media stream containing a plurality ofcomponent streams in a picture-in-picture mode. The picture-in-pictureinformation related to the media stream is obtained. According to suchpicture-in-picture information, a plurality of component streams isextracted from the media stream and is rendered in a picture-in-picturemode. An apparatus for implementing the method is also provided. Such anapparatus comprises an extractor and a processor. The extractorextracts, from the media stream, picture-in-picture information, and aplurality of component streams according to the picture-in-pictureinformation. The extracted component streams are processed in theprocessor for rendering in a picture-in-picture mode.

BRIEF DESCRIPTION OF THE DRAWINGS

The above features of the present invention will become more apparent bydescribing in detail exemplary embodiments thereof with reference to theattached drawings in which:

FIG. 1 illustrates a picture-in-picture display mode with one slavelayer picture overlaying on a master layer picture.

FIG. 2 shows the structure of an apparatus used to prepare, according tothe present invention, a media stream which contains multiple componentstreams for picture-in-picture purpose.

FIG. 3 shows one embodiment of the current invention to prepare a mediastream for use in a picture-in-picture application.

FIG. 4 shows the structure of a picture-in-picture rendering device usedto read a media stream containing picture-in-picture information andrender the stream in a picture-in-picture mode.

FIG. 5 shows the process of reading a media stream containingpicture-in-picture information and rendering the stream in apicture-in-picture mode according to the present invention.

DETAILED DESCRIPTION

In the present invention, a system level PIP solution is proposed byinserting the PIP information, such as the resolution/size, the locationand the aspect ratio of each component stream in a PIP mode, to the bitstream at system level, so that the receiver can dynamically operate thePIP without accessing the information from the decoder.

FIG. 1 illustrates the picture-in-picture display mode with two picturesources. The background picture is labeled as a master layer 110 and theforeground picture is labeled as a slave layer 120. In one embodiment,the location of the slave layer 120 is defined with respect to themaster layer 110 as a pair of offsets (a horizontal offset and avertical offset). The aspect ratio and the resolution of the slave layer120 are also determined according to the desired display region in themaster layer 110. Other parameters, such as alpha blending of the masterand slave layer pictures, can also be defined according to theapplication. The present invention can support multiple slave layers.Each layer can be defined in a similar way. In a different embodiment,the master layer and slave layers can be defined with respect to a fixedpicture/video called anchor picture/video so that the location, scalingand other PIP information are aligned for all layers. In the presentinvention, we refer to the content stream that consists of a masterlayer or a slave layer as a component stream. Note that a componentstream can be a video, an image, subtitles, or a base layer video, anenhancement layer video in SVC or one view of the video in MVC. In thefollowing, video component streams are used to describe the invention.The same principles apply to other component types.

FIG. 2 shows an apparatus for preparing a media stream which containsmultiple component streams for picture-in-picture purposes. The mediastream can be, for example, a program stream or a transport stream. Astream identifier 210 identifies the component streams 215 contained inthe media stream 205. A correlator 220 correlates those identifiedcomponent streams according to picture-in-picture relationships. SuchPIP relationships are determined by the application. In one application,a video of satellite images as a slave video may be, for example,associated/correlated with a video of a meteorologist commenting on theweather forecast as a master video. In a different application, thevideo of satellite images as a master video is associated with a videorecording a local town hit by a tornado as a slave video for educationpurposes. The correlator 220 also generates the PIP information 225based on the information of the master and slave component videos. ThePIP information 225 includes, but is not limited to,

-   -   Layer information that indicates the overlapping relationship        among different layers, such as which component video is the        master layer video, and which component video is the slave layer        video. For applications that support multiple slave layers, the        layer information also includes the overlapping relationship        among the slave layers.    -   Position information that indicates the position of each layer        when displayed in a picture-in-picture mode. In one embodiment,        the position information can be calculated with respect to the        master layer component video. In a different embodiment, the        position information is with respect to an anchor component        video.    -   Size information that indicates the size or resolution of each        layer when displayed a picture-in-picture mode. In one instance,        the size or resolution of a slave layer video is determined by        the size of the region in the master layer video that is        intended for displaying the slave layer video.    -   Aspect ratio information that indicates the aspect ratio of each        layer when displayed in a picture-in-picture mode. Similarly, in        one instance, the aspect ratio of a slave layer video is        determined by the aspect ratio of the region in the master layer        video that is intended for displaying the slave layer video.

Other information, such as alpha blending of the layers, can also beincluded.

The identified PIP information 225 may be inserted into the media streamby an embedder 240. In one embodiment, such embedding is realized byfeeding the PIP information 225 into a PIP information builder 230 togenerate PIP information data units 235. The information data units 235carry the same information as the PIP information 225, but conform tothe format of the component streams 215. The embedder 240 inserts theformat compliant PIP information data units 235 into the media streamalong with the component stream 215 so that the PIP information can beassociated with component streams to enable PIP at the receiver side.The output media stream 245 is different from the input media stream 205in that the output media stream 245 contains the PIP informationinserted by the embedder 240.

FIG. 3 shows the process of preparing a media stream which containsmultiple component streams for picture-in-picture applications. At step320, each component stream contained in the media stream retrieved atstep 310 is examined. First, the PIP relationship of the currentcomponent stream with other component streams is identified in step 330.Meanwhile, the PIP information as mentioned above is generated. Such PIPinformation is encapsulated into PIP information data units at step 340which are compliant with the format of the media stream. The PIPinformation data units are inserted into the media stream at step 350 tobe sent to the receiver.

In general, the PIP information for the component streams is stable fora certain period of time, such as within one scene of the video. Thatis, the value of position, size and other PIP information remain thesame for all the frames within that scene. However, as the content ofthe component stream for either the master layer or the slave layerchanges, the PIP information may need to be updated. One example of sucha change is that the topic of the director's commenting changes whichrequires moving the video of the director to a different position andwith a different size and aspect ratio. Another example could be thatthe source of the slave layer video switches to a different componentstream. Such changes of PIP information need to be updated and insertedinto the media stream. If PIP information data units are used forcarrying the PIP information, the PIP information data units should alsobe rebuilt based on the updated PIP information in compliant with theformat of the media stream and embedded into the media stream.

The present invention targets on supporting the multiple componentrelationship. It can be used in many multimedia applications such as theHTTP streaming and enriching the functionalities of HTML5. In thefollowing, two embodiments of PIP information data unit design aredisclosed for two streaming standards that shall support HTTP streaming:MPEG 2 Transport stream and ISO media format streaming. Since thecurrent versions of both standards do not support system level PIPimplementation, the proposed embodiments would be extensions to thecurrent standards. It would be appreciated that those skilled in the artwill be able to devise various versions of the PIP information dataunits that, although not explicitly described or shown herein, embodythe present principles and are included within its spirit and scope.

Listed in Table 1 is the proposed PIP information data unit for theMPEG-2 TS specification, referred to as a pip_descriptor. The descriptorcontains details of a video source's position, scaling and aspect ratioinformation. The syntax of the pip_descritor is explained as follows:

TABLE 1 PiP descriptor No. of Mnemo Syntax bits nic pip_descriptor( ) { descriptor_tag 8 Uimsbf  descriptor_length 8 Uimsbf   master_layer 8Uimsbf    layer_num 8 Uimsbf   horizontal_offset 16 uimsbf   vertical_offset 16 uimsbf   scale_info_present_flag 1 Uimsbf  if(scale_info_present_flag){       width 16 Uimsbf       height 16Uimsbf      }  aspect_ratio_info_present_flag 1 uimsbf  if(aspect_ratio_info_present_flag ) {    aspect_ratio_idc 4 uimsbf    if(aspect_ratio_idc = = Extended_SAR ) {    sar_width 8 uimsbf   sar_height 8 uimsbf     }     flexibility 2 bslbf     reserved 2bslbf   } }

-   -   descriptor_tag—The descriptor_tag is an 8 bit field which        identifies each descriptor in MPEG-2 standard. For more details        about descriptors in MPEG-2, see Section 2.6 of Information        technology—Generic coding of moving pictures and associated        audio information: systems, ISO/IEC 13818-1:2007.    -   descriptor_length—The descriptor_length is an 8 bit field        specifying the number of bytes of the descriptor immediately        following the descriptor_length field.    -   layer_num—This field indicates how a master layer component        video and slave layer component videos overlap. A value of 0 is        the lowest layer during overlay. In general, for one display, if        there is any overlap between layer i content with layer j        content and i<j, layer j content will be rendered for the        overlapped region. Using the parameter of layer_num, the support        for multiple-display devices can be easily implemented. In a        system with multiple displays/screens, such as using a TV as a        major display screen and an iPad as a secondary display screen,        the application can assign a higher probability for the content        in the component stream with higher layer_num value to be        rendered on a secondary display screen.    -   master_layer—This field indicates which component video is the        master layer of current component video layer. If the value of        master layer is same as the value of layer number, it means the        current component video is a master layer.    -   horizontal_offset—This field indicates the horizontal position        of the top left pixel of a current component video display        region with respect to the top left pixel of the master layer        component video or display region as shown in FIG. 1. This value        can also be measured with respect to an anchor component video.    -   vertical_offset—This field indicates the vertical position of        the top left pixel of a current component video display region        with respect to the top left pixel of the master layer component        video display region as shown in FIG. 1. This value can also be        measured with respect to an anchor component video.    -   width—This field is the horizontal size of a display region for        a current component video. Video frames of the current component        video are scaled to this size.    -   height—This field is the horizontal size of a display region for        a current component video. Video frames of the current component        video are scaled to this size.    -   aspect_ratio_info_present_flag—If the value of this field equals        to 1, it means that aspect_ratio_idc is present. If        aspect_ratio_info_present_flag equals to 0, then        aspect_ratio_idc is not present.    -   aspect_ratio_idc—This field specifies the value of the sample        aspect ratio of the luma samples in the component video. Table-2        shows the meaning of the code.

When aspect_ratio_idc indicates Extended_SAR, the sample aspect ratio isrepresented by sar_width and sar_height. When the aspect_ratio_idcsyntax element is not present, aspect_ratio_idc value shall be inferredto be equal to 0.

TABLE 2 Meaning of sample aspect ratio indicator Sample aspect Decimalaspect_ratio_idc ratio form 0 Unspecified 1 1:1(“square”)    1:1 2 10:11 0.9:1 3 32:30 1.066:1 4 59:54  1.09:1 5 4:3  1.33:1 6 3:2  1.5:1 7Extended_SAR

-   -   sar_width indicates the horizontal size of the sample aspect        ratio (in arbitrary units).    -   sar_height indicates the vertical size of the sample aspect        ratio (in the same arbitrary units as sar_width).    -   sar_width and sar_height shall be relatively prime or equal        to 0. When aspect_ratio_idc is equal to 0 or sar_width is equal        to 0 or sar_height is equal to 0, the sample aspect ratio shall        be considered unspecified.    -   flexibility indicates the PIP behavior for a player. When it is        set to 0, it is a type of hard coded PIP and means player cannot        alternated it or remove it which can be used in applications        such as none removable Ads. When the flexibility field is set to        1, player can alternate the PIP according to an algorithm such        as screen size and point-of-interest mask layer etc. When the        flexibility field is set to 2, player allows user to change the        PIP and even turn off PIP, which can enhance the interactivity        experience and also reduce the control overhead.

In a picture-in-picture mode, the target_background_descriptor andpip_descriptors are used to place the video at a specific location inthe display as shown in FIG. 1. The target_background_descriptor isexplained in more detail in section 2.6.12 of Informationtechnology—Generic coding of moving pictures and associated audioinformation: systems, ISO/IEC 13818-1:2007. The pip_descriptor shall beincluded in the descriptor loop of the program map table/section (PMT)defined in Section 2.4.4.8 of Information technology—Generic coding ofmoving pictures and associated audio information: systems, ISO/IEC13818-1:2007. When the updating of PIP information is necessary, thepip_descriptor can be dynamically updated along with the PMTinformation.

For ISO media file format, two PIP information boxes are proposed, onefor live updating and one for offline updating. The live updating PIPinformation box is used during live streaming, wherein the PIPinformation box is included in the metadata portion of the componentstream and can be updated constantly. The offline updating PIPinformation box is used with offline applications such as DVD, wherebythe PIP information is only sent once in the box and the information isinserted in the data sample portion of the stream. When the PIPinformation changes, the PIP information box is updated and insertedinto the current data sample portion of the stream.

1. PIP information box for the live updating

-   -   Definition:    -   Box Type: ‘pipl’    -   Container: Track Fragment Box (‘traf’) or Track Box (‘trak’)    -   Mandatory: No    -   Quantity: Zero or one    -   Syntax:

 aligned(8) class PiPInfoBox extends  Box (‘pipl’) {  template int(16)layer_num = 0;  template int(16) master_layer = 0;  unsigned int(32)horizontal_offset;  unsigned int(32) vertical_offset;  unsigned int(32)width;  unsigned int(32) height; }

-   -   Semantics

Each entry is the same as explained before.

2. PIP information box for the offline updating

-   -   Definition:    -   Box Type: ‘pips’    -   Container: Sample Table Box (‘stbl’)    -   Mandatory: No    -   Quantity: Zero or one    -   Syntax:

 aligned(8) class PiPInfoBox extends  Box (‘pips’) {  unsigned int(32)entry_count;  int i;  for (i=0; i < entry_count; i++) {  unsignedint(32) pip_sample_number;  template int(16) layer_num = 0;  templateint(16) master_layer = 0;  unsigned int(32) horizontal_offset;  unsignedint(32) vertical_offset;  unsigned int(32) width;  unsigned int(32)height;  } }

-   -   Semantics    -   entry_count—is an integer that gives the number of entries in        the following table. Each entry corresponds to a component        stream in the media stream. pip_sample_number—gives the number        of the samples for which that are PIP points in the stream. The        details about the definition of a sample can be found at Section        3.1.10 of Information Technology—Coding of Audio-Visual        Objects—Part 12: ISO base media file format.

At the receiver side, in order to render the picture-in-picture for amedia stream that contains multiple component streams, PIP informationis extracted from the media stream. According to the PIP information, aplurality of component streams, including a master layer componentstream and one or more slave layer component streams, are determined andextracted from the media stream. The component streams, including themaster layer and slave layer component streams, are rendered accordingto the PIP information. In some cases, for example when the videocontent in the component stream has different size/resolution and aspectratio from the required size and aspect ratio for PIP, the componentstream needs to be pre-processed before rendering. Such a pre-processingrelates to a scaling operation of the component streams. Otherpre-processing would be necessary depending on the component streams andthe PIP information. The rendering process will constantly monitor thePIP information, and make changes whenever PIP information has updates.

FIG. 4 shows the structure of a PIP rendering device used to read amedia stream containing PIP information. Extractor 410 extracts PIPinformation from the input media stream, and further extracts a mastercomponent stream and all slave component streams from the media streamaccording to the PIP information. A processor 430 processes theextracted component streams according to the picture-in-pictureinformation to generate output to a display 440. In one embodiment,extractor 410 extracts PIP information data units, such aspip_descriptors or PIP information boxes from the media stream and ananalyzer 420 is employed to analyze the extracted PIP information dataunits. One output from the analyzer 420 is the references to the masterand slave layer component video streams. With those references, theextractor 410 is able to extract the necessary component streams for theprocessor 430. Analyzer 420 further parses the PIP information dataunits to generate PIP information that is understandable to theprocessor 430. The processor 430 can also accept outside controlsignals, e.g. from viewers, for controlling the PIP effect, such asenabling or disabling the PIP feature. The control signal can containthe desired position or size information for the component streams. Suchinformation overwrites the corresponding information provided in PIPinformation from the media stream. The processor 430 may comprise avideo decoder which decodes the extracted master and slave componentstreams. The decoding of master and slave layer component streams can beperformed in parallel. If the pre-processing as mentioned before isnecessary for the component streams, the processor 430 further performssuch pre-processing by calling its pre-processor unit before generatingthe PIP video for displaying at display 440. Display 440 may alsoinclude more than one display screen for multi-display applications asdiscussed before.

FIG. 5 shows a process for reading a media stream containing PIPinformation and rendering the stream in a PIP mode according to thepresent invention. PIP information data units such as pip_descriptors orPIP information boxes are extracted in step 510. PIP information isfurther identified in step 520 by analyzing the extracted PIPinformation data units, such as which streams are the master and slavelayer component streams. With the PIP information, the process is ableto extract the required component streams from the media stream in step530. For each of the component streams, a decision step 550 determineswhether the component stream requires pre-processing. If pre-processingis needed, the component stream content is processed according to thePIP information in step 560, such as resizing the content to a desiredsize and an aspect ratio. If pre-processing is determined to beunnecessary, the process moves directly to step 570 for rending. Theloop for each of the component streams may be realized in parallel usingmulti-core processors or through multi-thread implementations.

Although preferred embodiments of the present invention have beendescribed in detail herein, it is to be understood that this inventionis not limited to these embodiments, and that other modifications andvariations may be effected by one skilled in the art without departingfrom the scope of the invention as defined by the appended claims.

The invention claimed is:
 1. A method for preparing a media stream forpicture-in-picture applications, said media stream containing aplurality of component streams, the method comprising: generatingpicture-in-picture information from the plurality of component streamscontained in the media stream, wherein said picture-in-pictureinformation comprises position information, said position informationindicating the position of a component stream when displayed inpicture-in-picture mode; and inserting the picture-in-pictureinformation into the media stream.
 2. The method of claim 1, furthercomprising updating the picture-in-picture information according tochanges in the plurality of component streams.
 3. The method of claim 2,wherein the inserting step comprises building picture-in-picture dataunits according to the generated picture-in-picture information; andinserting the picture-in-picture data units into the media stream.
 4. Anapparatus for preparing a media stream for picture-in-pictureapplications, said media stream containing a plurality of componentstreams, the apparatus comprising: a correlator for generatingpicture-in-picture information from the plurality of component streamscontained in the media stream, wherein said picture-in-pictureinformation comprises position information, said position informationindicating the position of a component stream when displayed inpicture-in-picture mode; and an embedder for inserting thepicture-in-picture information into the media stream.
 5. The apparatusof claim 4, wherein the media stream is one of a transport stream and aprogram stream.
 6. The apparatus of claim 5, further comprising apicture-in-picture information builder for generating picture-in-pictureinformation data units from the identified picture-in-pictureinformation, the picture-in-picture information data units being fedinto the embedder for inserting into the media stream.
 7. A method forrendering a media stream in a picture-in-picture mode, said media streamcontaining a plurality of component streams, the method comprising:obtaining picture-in-picture information related to the media stream,wherein said picture-in-picture information comprises positioninformation, said position information indicating the position of acomponent stream when displayed in picture-in-picture mode; extracting aplurality of component streams from the media stream according to theobtained picture-in-picture information; and rendering the plurality ofcomponent streams according to the obtained picture-in-pictureinformation.
 8. The method of claim 7, wherein the plurality ofcomponent streams comprises a master layer component stream and at leastone slave layer component stream.
 9. The method of claim 7, furthercomprising if necessary, pre-processing at least one of the extractedplurality of component streams according to the picture-in-pictureinformation prior to rendering.
 10. The method of claim 9, wherein theobtaining step comprises extracting picture-in-picture information dataunits from the media stream; and analyzing the picture-in-pictureinformation data units to identify the picture-in-picture information.11. The method of claim 7, wherein the picture-in-picture informationcomprises information of layer, size, aspect ratio of the componentstreams in the picture-in-picture mode.
 12. An apparatus for rendering amedia stream in a picture-in-picture mode, said media stream containinga plurality of component streams, the apparatus comprising an extractorfor extracting, from the media stream, picture-in-picture information,and for extracting a plurality of component streams according to thepicture-in-picture information, wherein said picture-in-pictureinformation comprises position information, said position informationindicating the position of a component stream when displayed inpicture-in-picture mode; a processor for processing the extractedplurality of component streams according to the picture-in-pictureinformation.
 13. The apparatus of claim 12, wherein the extractorextracts picture-in-picture information data units from the mediastream, the apparatus further comprising an analyzer for analyzing thepicture-in-picture information data units to get picture-in-pictureinformation, and for providing references to the plurality of componentstream to the extractor.
 14. The apparatus of claim 12, wherein thepicture-in-picture information further comprises at least information oflayer, size, and aspect ratio of the component streams in thepicture-in-picture mode.
 15. The apparatus of claim 12, wherein theprocessor comprises a pre-processing unit for scaling the at least onecomponent stream to a size specified in the picture-in-pictureinformation.
 16. The method of claim 3, wherein the picture-in-picturedata units comprise picture-in-picture descriptors, and the insertingstep further comprises including the picture-in-picture descriptors intoa program map table of the media stream.
 17. The method of claim 1,wherein the picture-in-picture information further comprises informationof layer, size, aspect ratio of the component streams in thepicture-in-picture mode.
 18. The apparatus of claim 6, wherein thepicture-in-picture data units comprise picture-in-picture descriptors,and the embedder further performs steps of including thepicture-in-picture descriptors into a program map table of the mediastream.
 19. The apparatus of claim 4, wherein the picture-in-pictureinformation further comprises at least information of layer, size, andaspect ratio of the component streams in the picture-in-picture mode.